Skip to main content

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: speederk Date: Oct 30, 2012 5:42pm
Forum: web Subject: Re: robots.txt

Here is another issue I would like to comment on.

I am trying to do a little research on an electronics company (Harmon Electronics ). They HAD a web page but the company was bought out by GE and thus changed names.

When their domain expired and a web squatter took the name over and placed a robots.txt block on the same address.

Now, trying to find the electronics companies archive is not available.

In order to keep IA motto accurate of "Universal access to all knowledge" it would be in the best interests of IA to scrub the databases and find out where the robots.txt file was implemented and allow access to prior work...if it still exists.

This post was modified by speederk on 2012-10-31 00:42:45